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Though debates continue over the proper role of the federal government in many aspects of education, there 
is a broad consensus that federal investment in education research, development, and dissemination is vital for 
advancing the abiiity of the field to improve outcomes for all students. 

This consensus should result in greater federal attention and increased investment, yet compared to other 
cabinet-level agencies, the Department of Education (ED) has the fewest resources dedicated to R&D. As a 
result, investments have not kept pace with emerging, urgent student needs. Furthermore, the fragmentation 
of research activities among agencies and legislation inhibits the impact of existing federal dollars. The R&D 
functions in education are not confined to the Institute for Education Sciences (lES) and are not exclusively 
contained in the authorizing legislation most responsible for those investments, the Education Sciences Reform 
Act (ESRA). For exampie, the Comprehensive Centers Program is housed within the Office of Elementary and 
Secondary Education within ED, whiie the Investing in Innovation (13) Program, funded through the American 
Recovery and Reinvestment Act (ARRA), exists within the Office of Innovation and Improvement. 

To catalyze innovation in education R&D, the Obama Administration has proposed to create a new unit of ED, 
called ARPA-ED, that would be analogous to the high-profile Defense Advanced Research Projects Agency 
(DARPA) in the Department of Defense. Since ESRA was enacted over ten years ago, lES has made great 
strides in upgrading the quality of education research. Despite these promising advances, the current federal 
approach to education R&D — too iittie investment, scattered across too many programs, with too iittie attention to 
dissemination — continues to both undermine and underestimate the capacity of the American education system 
to meet the pressing needs of students and of the country. The pending reauthorization of ESRA creates new 
opportunities to better harness the tremendous research capacity we have in America to turn broad consensus 
into broad benefit for our ever-more diverse schools and students. 

In February 2013, the Aspen Institute Education & Society Program held a seminar with our network of senior 
Congressional education staff to discuss these issues. For the meeting, we commissioned a paper from education 
writer Nancy Kober on the history of the federal role in education research, and we developed a set of maps that 
provide an approximate view of the federal investment in education R&D. From those conversations, and many 
more with leaders In the policy, research, and practice community, we saw an opportunity to elevate and inform 
discussions on this important set of issues. So we refined these resources to share them more widely. We also 
offer new essays from a range of expert voices. Tim Knowles and Jens Ludwig, Vivian Tseng, Robert Slavin, and 
Michael Petrilli bring diverse perspectives to provoke new thinking about how the federal government can best 
leverage its resources to improve education research and development, and to ensure that the research makes 
its way to schools and classrooms for the benefit of students. We organize their essays to represent a spectrum 
from modest yet high-impact policy revisions to a total reimagining of the current research-to-practice pipeiine. 

We are hopeful that these resources contribute to a more robust discussion and to a renewed focus on the 
importance of getting high-quaiity, timely research into the hands of those throughout the system who can 
translate it into improved outcomes for ail students. 
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A Brief History of Federal Efforts to 
Improve Education Research 

Nancy Kober 

Freelance Reporter and Education Consultant 


Main Themes: Similar Criticisms, Similar Strategies 

Over the past six decades, Congress and the Executive Branch have taken a variety of actions to deveiop, expand, 
and improve a system of federai support for research in education. This paper describes the main federai iegislative 
and administrative efforts undertaken in past decades to achieve this goai and the common themes that underiie these 
actions. The information is drawn from the iegislative history and from analyses by individual scholars, policy experts, 
research groups such as the National Research Council, journalists, and others. 

Similar criticisms of the federal role in educational research have been raised by multiple sources across several 
decades. Three criticisms in particular have come up time and again: 

• The federal role lacks clarity and coherence of purpose. 

• Federally funded research has problems with quality, usefulness, timeliness, and relevance to the classroom. 

• The federal research agenda has been vulnerable to political and ideological influences. 

Although policymakers have tried several approaches over the years to address these criticisms and shape the federal 
research agenda. Congressional and Administration leaders have relied most heavily on two strategies: 

• Creating new federal agencies and federally supported research entities or restructuring existing ones 

• Cutting or increasing annual budgets for education research and earmarking or restricting funds for specific 
activities 

These themes weave throughout the history of federal efforts to improve education research. The key events and 
examples described below are intended to give a broad overview of this history; many other significant actions and 
examples have been omitted for brevity’s sake. 

Origins: Coiiecting Vitai Education Statistics 

The main reason why the federal government became involved in education in the first place was to fulfill a need for 
national data on education. The mission of the first Department of Education, according to its 1867 authorizing law, 
was to “collect such statistics and facts as shall show the condition and progress of education in the several States 
and Territories” and disseminate information about schools and teaching that would “aid the people of the United 
States in the establishment and maintenance of efficient school systems.” 

Data collection remained the chief activity of the Department (later the Office and then the Bureau) of Education for the 
next 90 years and continues today through the work of the National Center for Education Statistics (NOES). During 
these early years, research beyond collecting statistics was not considered a federal priority. 
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Summary Table: Key Federal Efforts to Support Education Research 


EVENT 

YEAR INITIATED 

AUTHORIZING LEGISLATION 

PURPOSE 

Federal collection of 
education statistics 

1867 

An Act to Establish a 
Department of Education 
(14 Stat. 434) 

Collect statistics on the 
condition and progress of 
education 

Federal funding for 
research projects 

1954 

Cooperative Research Act 
(RL. 83-531) 

Support field-initiated studies, 
primarily at universities 

Education research and 
development centers 

1964 

Created by executive authority 
under Cooperative Research 
Act; codified in Elementary and 
Secondary Education Act of 
1965 (RL 89-10) 

Conduct large-scale basic 
and applied research in 
specific education fields 

Regional educational 
laboratories 

1965 

Elementary and Secondary 
Education Act 
(RL 89-10) 

Translate research into 
practice by conducting 
applied research, 
disseminating research 
findings, and providing 
technical assistance 

National Institute of 
Education 

1972 

Education Amendments of 
1972 (RL 92-318) 

Strengthen leadership 
and improve the quality of 
education research 

Office of Educational 
Research and Innovation 

1979 

Created by Department of 
Education Organization Act 
(RL. 96-88); reorganized in 
1 985 by executive action; 
reauthorized in 1 994 by Goals 
2000: Educate America Act 
(RL 103-227) 

Rrovide federal leadership and 
strengthen the educational 
research and development 
system; support high-quality 
research; improve educational 
techniques and training 

Institute for Education 
Sciences 

2002 

Education Sciences Reform Act 
(RL 107-279) 

Improve federal research, 
statistics, and evaluation; 
promote independence and 
scientific rigor of federally 
funded research 


The 1950s: Federal Support for Field-Initiated Research 

The federal government began taking a more active role in education research in 1 954 with passage of the Cooperative 
Research Act. This legislation made federal funds available for research projects proposed by universities, state education 
departments, and other entities. But this field-initiated approach “afforded little opportunity for federal officials to shape 
a national research agenda” and “eventually led to concerns about the fragmented and noncumulative nature” of the 
funded projects, according to Research and Education Reform, a 1 992 report by the National Research Council (NRC). 
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The 1960s: Centers and Laboratories 


In 1 964, the U.S. Office of Education under President Johnson sought to develop a more coherent means of supporting 
large-scale, long-term research on critical educational problems by establishing education research and development 
centers. Between 1964 and 1966, eleven such centers were created. In the absence of federal guidance on research 
priorities, each center proposed its own specialized areas of research. In 1965, Congress created a network of regional 
educational laboratories to develop and disseminate new curricula and teaching methods and adapt research for 
use in classrooms. By 1969, there were twenty laboratories serving designated regions of the country. In the mid 
1 960’s, the Office of Education also created the Educational Resources Information Center (ERIC) to provide access 
to research information. 

Appropriations for the federal centers and labs grew rapidly at first, although not to the level envisioned by their 
Congressional sponsors. Some center and lab products had a clear impact. A notable example was a seminal 
federally funded study — Equality of Educational Opportunity, better known as the Coleman report — published in 1 966 
by James Coleman and colleagues at a Johns Hopkins research center. This study marked an early attempt to 
document achievement gaps between black and white children and launched an ongoing debate about the impact 
of family background on disparifies in educational outcomes. To cite another example of impacf , the centers and labs 
supported influential research in cognitive science and produced innovative programs for mathematics instruction that 
were used by thousands of students. 

During their first several years of operation, the centers and labs had difficulty fulfilling their ambitious goals. Key 
policymakers and others charged that the roles of these research entities were fuzzy and their work was fragmented, 
lacked value, and had not improved practice as much as intended. As a result of this dissatisfaction and other federal 
budget pressures, Congress cut funding for the lab program substantially, and fourteen labs were closed. Federal 
funding for the centers dropped from $14.7 million in fiscal year 1968 to $10.7 million in fiscal year 1970, and the 
number of centers was reduced to eight. 

In the subsequent decades, the centers and laboratories have weathered ups and downs in funding and have 
undergone several changes in mission, number, and structure. They continue to play an important role in the federal 
education research landscape and receive about 10 percent of the current Department of Education research budget. 

The 1970s and Early 1980s: National Institute of Education 

The Nixon Administration and key Members of Congress of both parties recognized the need to legitimize and upgrade 
the quality of federally supported education research. In 1972, the National Institute of Education (NIE) was created 
within the Department of Health, Education, and Welfare (HEW). The Insfifute inherited staff and programs from the 
HEW’s Office of Education and from the Office of Economic Opportunity, the lead agency for anti-poverty programs. 
NIE’s mission was to provide the federal leadership needed to build an effective system of educational research and 
development. It also had a mandate to expand knowledge about promoting equity. 

The Institute was intended to draw the best scientific talenf to the study of important issues in education, much as the 
National Institutes of Health and National Science Foundation did for their fields. To help achieve this goal, NIE was 
given a degree of independence to hire a permanent staff of scholars outside the civil service rules and to contract 
with other researchers. 

“From its inception, NIE was rarely free of turmoil,’’ the 1992 NRG report observed. “Inadequate levels of funding 
hampered NIE’s abilifyto sponsor major, long-term research projects.’’ In just one year, between 1973 and 1974, the NIE 
budget was cut in half — from $136 million to $65 million — and never exceeded $80 million for the rest of ifs existence. 
The Institute had six directors and four acting directors in 1 3 years, which “did not allow strong, consistent leadership 
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to be established,” according to the NRC. During NIE’s iifespan, it also underwent several internal reorganizations. 
Most notabiy, when the U.S. Department of Education was created in 1979, NIE was placed within a new Office of 
Educational Research and Improvement (OERI) but retained a fair degree of autonomy. 

In this climate of limifed funding and organizational churn, NIE tried, with varying degrees of success, fo respond to a 
host of competing demands. These included conducting research that was relevant to practitioners and disseminating 
the results; evaluating federal programs; fulfilling ifs equity mission through work focused on the needs of poor, minority, 
limited English proficient, and female students; and studying fundamental issues in education. 

Among the more influential efforts supported by NIE were studies of the characteristics of effective schools, 
which helped to spawn the school reform movement, and research on reading skills, which culminated in the 
popular 1 984 report Becoming a Nation of Readers. Research on early childhood education, career education, and 
school finance also helped NIE gain credibility among some knowledgeable stakeholders, as did a Congressionally 
mandated evaluation of Title I of fhe Elementary and Secondary Education Act and a mandated study of school 
violence and vandalism. 

Through much of ifs existence, NIE faced ongoing efforts by the Administration and Congress to manage its priorities 
or exert political or ideological influence over its research agenda. In the early 1980s, for example. President Reagan 
fired all 1 5 members of NIE’s policymaking board, despite their staggered statutory terms, in the interest of moving the 
agency toward a more conservative research agenda. The newly appointed NIE director sent a letter to the President 
asserting that the federal government has no role in conducting education research, provoking a controversy that 
ended with the director’s resignation. Members of Congress, for their part, sought to shape the Institute’s agenda 
by earmarking funds for specific projects. When these earmarks were added to the funding reserved for the regional 
educational labs, centers, and the ERIC system, this left limited funding for new NIE initiafives or core priorities. 


The late-1980s and 1990s: The Office of Educational Research and Improvement 

In 1985, Secretary of Education William Bennett abolished NIE, NCES, and the other semi-autonomous entities within 
OERI, citing his desire to eliminate superfluous layers of bureaucracy. The functions of these entities were merged into 
a reorganized OERI with offices for research, statistics, improvement of pracfice, and two other areas. Initially, this 
approach was “opposed by most education policymakers and researchers as unnecessary and a downgrading of the 
importance of research,” according to a 2001 analysis by University of Michigan professor Maris Vinovskis. 

Overtime, OERI was beset by many of the same problems as NIE. The 1992 NRC study of OERI “found an agency that 
has been challenged by several external difficulties over which it has little control, as well as several internal problems. 
The external problems begin with [...] controversies about the appropriate roles for federal education research and 
development. There have been attempts to make OERI serve political purposes, and the agency has been given 
marginal discretion over new initiatives. The agency has also been inadequately funded.” As a result, OERI had difficulty 
pursuing and disseminating the type of sustained, high-quality research that was likely to have an impact on critical 
education issues. 

OERI could point to some accomplishments. In the 1 980s and 1 990s, the National Assessment of Educational Progress 
(NAEP) sponsored by NCES grew in technical sophistication and came to be seen as a respected source of national 
data on student achievement. NCES also improved its ability to produce high-quality data reports in a more timely 
way. Many innovative R & D programs were developed, piloted, and expanded with federal support; examples include 
computer-based instruction, research on student motivation, and programs to develop higher-order thinking skills. In 
addition, OERI-sponsored evaluations of federal programs had an impact on legislative actions. For example, a 1989 
national assessment of vocafional educafion poinfed to the lack of targeted funding for disadvantaged students and 
the need to improve program quality. Influenced by these and other findings. Congress enacted amendments to the 
vocational education legislation that focused federal support on districts with concentrations of poor studenfs, on 
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programs that integrate academic and vocational instruction, and on effective programs that produce the desired 
results, according to the 1992 NRC report. 

At the same time, “only a few lines of research [were] sustained for the time needed to bring them to fruition,” the NRC 
concluded. “Few efforts are undertaken to synthesize and publicize what the agency has learned and accomplished. 
Quality control is uneven, and the agency rarely attempts to resolve debates on important issues of education research.” 

Like its predecessor agency, OERI was pressured to take on activities beyond its core mission. One such example 
occurred in the 1 990s, when the Clinton Administration used research funding to advance the work of the National 
Education Goals Panel and pursue development of national voluntary tests. This decision was criticized by some 
lawmakers and was ultimately overturned by Congress. 

Policymakers of both political parties continued to use the legislative process to direct the Department of Education 
to spend research and development funding on specific activities. An example was a 1 997 amendment sponsored by 
Congressmen David Obey (D-WI) and John Porter (R-IL) to provide federal funds to implement models of comprehensive 
school reform, such as the Success for All approach originally developed at a federally funded R&D center. 

The OERI reauthorization of 1994, based on a proposal by Congressman Major Owens (D-NY), created a semi- 
independent advisory board for OERI as a way to shield the agency from political influence. The legislation also 
reorganized OERI around multiple institutes, each devoted to a specific educational problem. But despite growth in 
OERI’s budget during the George H.W. Bush and Clinton Administrations, the sizeable commitment needed to get the 
institutes off the ground was never provided, and the plan was abandoned. 

The 2000s and 2010s: Institute of Education Sciences 

The most recent federal overhaul of the education research agency occurred in 2002, when Congress replaced OERI 
with a new Institute of Education Sciences. lES consists of three National Centers for research, statistics, and evaluation 
and regional assistance, plus a special education research center moved from the Office of Special Education and 
Rehabilitative Services. 

One goal of this restructuring was to further buffer the agency from political influence by increasing its autonomy. 
Toward this end, lES is headed by an independent director with a six-year term and a higher rank than previous OERI 
directors. A new independent advisory board — a majority of whom must be researchers — has the power to formally 
approve the agency’s long-term priorities and peer review process. 

Another goal of the lES was to strengthen the quality and scientific rigor of federally supported research. This was 
manifested by provisions in the lES authorizing law (as well as in the No Child Left Behind Act and other statutes) that 
require the Institute to support “scientifically based” research. 

Since 2002, the overall budget for lES has increased, despite some interim years of flat funding. In 2012, close 
to $600 million was appropriated for lES. This amount includes funding for NCES and the other national Centers, 
as well as for the regional labs and R&D centers and other major activities. A 2006 recompetition of the regional 
educational laboratory contracts led to new operators for some labs and a shift in mission for all labs toward more 
slow, rigorous research. 

During its ten-year existence, lES has demonstrated independence on several occasions. For example, as Dickinson 
College professor Andrew Rudalevige noted In a 2009 article, the lES director “was careful to cast caveats upon 
education secretary Margaret Spellings’s claims that rising student test scores in 2005 offered ‘proof’ of NCLB’s 
effectiveness, adding in 2007 that widely divergent state standards for ‘proficiency’ in fact made it hard to judge 
NCLB’s impact.” Congress still has some control over the research agenda through budget and legislative decisions. 
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A report by the National Board for Education Sciences, the independent policy board for lES, cited improvements 
made by the Institute during its first five years in the “rigor, relevance, and accessibility of federally funded research,” 
while noting that much remains to be done to institutionalize and build on the gains. In a 2008 biennial report to 
Congress, “Russ” Whitehurst, the first lES director, pointed to numerous interventions that lES has found to make a 
difference in student outcomes, such as the Accelerated Middle Schools program. 

lES has not been immune to controversy. For example, Whitehurst placed considerable emphasis on randomized, 
controlled trials as the “gold standard” of rigorous research methods, a position that some researchers and stakeholders 
felt was too narrow. This stance has softened somewhat under current director John Easton. Some lawmakers have 
also raised questions about what they see as a lack of lES-supported research on major challenges and policy issues 
facing education, such as teacher quality and teacher evaluation and the effects of the No Child Left Behind Act. 

“Much progress has been made over the past nine years toward establishing lES as a significant contributor to 
foundational knowledge,” concluded a 201 1 report by the American Educational Research Association. The report 
added that “relatively few proposals for modification of the current legislation can enhance even further the contributions 
of this small agency to the nation’s well-being.” 

Conclusion 

The past six decades of federal support for education research could be characterized as a series of attempts to 
resolve the same fundamental questions. What is an appropriate mission for federal research agencies? How can the 
federal government support research that addresses the big questions in education and has an impact on teaching 
and learning? How can federal leadership improve the quality and reputation of the education research enterprise? 

Over the years, policymakers have used similar strategies to address these questions. Chief among them is the 
strategy of changing the organization, autonomy, placement, and mission of the main federal agency that oversees 
research. But this has sometimes come at a price of unstable leadership, competing responsibilities, and the turmoil 
and delays associated with any reorganization. 

Administration leaders and Congress have also used their executive, legislative, and budget powers to shape — 
some might say micromanage — ^the research agenda. While these strategies have often been used to address clear 
problems, they have also been applied to achieve political or ideological ends. The result, at times, has been a lack 
of coherence in the research agenda, energy spent on political battles, and unstable funding — in amounts that many 
would argue still fall short of what is needed to produce high-quality research with a demonstrated impact. 

Even so, progress has been made overtime in protecting the independence of the federal research agency, improving its 
reputation for scientific inquiry, and producing some studies that have changed education. The pending reauthorization 
of the education research authorizing legislation offers an opportunity to build on these accomplishments and address 
persistent problems. 
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WHICH FEDERAL AGENCIES CONDUCT 
RESEARCH RELEVANT TO EDUCATION? 

A SAMPLE 


A constellation of federal agencies engages in research, development, and dissemination activities that impact 
the field of education, yet there is no one catalog of these activities. This map takes a step in that direction by 
offering a sample of the agencies and programs most involved in the development and dissemination of education 
research. It is necessarily a rough snapshot: There is no clear definition of a federal education program, much 
less a definition of one with a full or partial orientation toward research.* The programs represent activities beyond 
the formal Office of Management and Budget definition of “research and development,” which is too constrictive 
to capture many federal activities that meaningfully inform practices for student learning. The map thus includes 
activities that influence but do not focus explicitly on K-1 2 education, as well as high-impact program evaluation, 
statistical collection, and technical assistance activities. 

U.S. Department of Education (ED) 

In addition to high-profile grant programs like the Race to the Top Fund and Investing in Innovation (i3), ED 
supports and conducts research, development, and dissemination through a broad swath of programs. National 
research centers on disabilities and career and technical education complement 22 state-focused regional and 
content centers under the Comprehensive Centers Program. ED also partners extensively with other agencies, 
but its main vehicle for driving research is its research unit, the Institute of Education Sciences (lES). 

Institute of Education Sciences (lES) 

National Center for Education Statistics (NCES) 

NCES is the primary federal entity for collecting and analyzing data related to education. In addition to providing 
funding for PISA assessments, NCES houses the National Assessment of Educational Progress (NAEP) and 
National Assessments of Adult Literacy (NAAL), and it oversees state grants for longitudinal data systems. 

National Center for Education Research (NCER) 

NCER research programs are designed to produce research that is scientifically rigorous and relevant to the 
needs of education practitioners and decision makers. NCER also funds pre-doctoral and postdoctoral research 
training programs to invest in the training and development of the next generation of education researchers. 

National Center for Education Evaluation and Regional Assistance (NCEE) 

NCEE conducts unbiased large-scale evaluations of education programs and practices supported by federal 
funds; provides research-based technical assistance to educators and policymakers; and supports the synthesis 
and the wide spread dissemination of the results of research and evaluation throughout the United States. In 
addition to administering regional education lab contracts, NCEE houses the What Works Clearinghouse. 

National Center for Special Education Research (NCSER) 

Sponsors a comprehensive program of special education research designed to expand the knowledge and 
understanding of infants, toddlers and children with disabilities. It oversees six National Special Education Research 
Centers and works with the other three lES centers to develop and evaluate services for students with disabilities. 
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National Science Foundation (NSF) 

NSF engages in education research primariiy through the Education and Human Resources Directorate (EHR, 
see below), but smaller programs of research are also housed in partner directorates. Under the President’s 
EY 2014 budget proposal, 1 14 of 220 programs conducting STEM education activities would be consolidated 
under three agencies: the Department of Education, NSE, and the Smithsonian Institution. 

Education & Human Resources Directorate 

The Education and Human Resources Directorate’s (EHR) consists of four divisions: Graduate Education (DGE), 
Research on Learning in Eormal and Informal Settings (DRL), Division of Research on Learning in Eormal and 
Informal Settings (DRL), and Human Resource Development (HRD). DRL is the lead division on research in STEM 
learning. Due to NSE’s focus on undergraduate and graduate STEM work, a number of programs in EHR and 
its partner directorates allocate small amounts each to the study of why students select particular majors and 
strategies for recruiting students to STEM studies. 

Department of Defense (DOD) 

DOD conducts extensive basic and applied research relevant to education among its various armed services 
branches, a small fraction of which is represented in the map. DOD has dedicated funding to the Advanced 
Distributed Learning Initiative (ADL), which works to develop the state of the art in education and training through 
the use of technology and innovative learning methodologies. DOD also has done significant work in educational 
technology through the Office of Naval Research and its peer offices in other armed forces branches, as well as 
under DARPA, focusing heavily on “intelligent” tutoring models. DOD is also collaborating more heavily in recent 
years with ED on joint STEM-oriented projects. 

Health and Human Services (HHS) 

HHS engages in many programs of R&D relevant to education, but most notable are ongoing evaluations of the 
Head Start pre-K program and work in the National Institutes of Health (NIH). The Eunice Kennedy Shriver National 
Institute of Ghild Health and Human Development (NIGHD), under NIH, supports basic and translational research 
into how children develop school readiness skills, reading skills and related disabilities, and math and science 
cognition and related disabilities. These basic and translational scientific studies further our understanding of the 
genetic, neurologic, cognitive, behavioral, and social-emotional aspects of learning and reasoning — from both 
a normative and pathological developmental perspective — and provide the evidence base for a broad range of 
researchers to understand how children learn and grow, including those supported by other federal agencies, as 
the basis for more applied education research. 

* Government Accountability Office, Overview ofK-12 and Early Childhood Education Programs, cover letter, 201 0, www.gao.gov/new.items/d1 051 .pdf. 

' U.S. Dept, of Education, FY201 3 Budget Request to Congress by Program, http://www2.ed.gov/about/overview/budget/budget13/summary/13pbapt.pdf. 

2 National Science Foundation, FY201 3 Budget Request, http://www.nsf.gov/about/budget/fy2013/toc.jsp. 

3 NSF Education & Human Resources Directorate, FY2014 Budge Request, http://www.nsf.gov/about/budget/fy2014/pdf/12_fy2014.pdf. 

“ Only one armed forces branch represented for sample. Senior staff of represented offices, Department of Defense, November 201 3. 

^ NICHD, FY201 3 Budget, http://www.nichd.nih.gov/about/budget/CJ/Documents/CJ_NICHD_2013.pdf. 

0 American Association for the Advancement of Science. FY201 3 Appropriations Summary Table, http://www.aaas.org/spp/rd/fy201 3/total1 3c.pdf. 
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Comprehensive 
Centers Program 
$51. 1m 


The National Research 
Center for Career and 
Technical Education 
(NRCCTE) 

$7.8m 


Fund for the Improvement 
of Postsecondary Education 
(FIPSE) 

$3.5m 

Investing in 
Innovation (13) ^ 

$ 149.4m 

Research in Education 
& Learning 
(REAL) f 

$59.4m 

Discovery Research 

K-12 ^ 

$1 14.2m 


Promoting Research & 
Innovation in 
Methodologies for 
Evaluation 
(PRiME) 

$7m 


Cyber Learning: Transforming Education 

$18m 

(across 3 Directorates) 

Total Non-EHR 
Research* 
$5.7 bil 


National Institute on Disability and 
Rehabilitation Research (NiDRR) 

$ 108.8m 


Statistical Collection 

^ $ 108.7m 
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Office of Special Education & 
Rehabilitative Services 
_ (OSERS) 

$16.4 bil 


Office of Elem. & Secondary Education 
(OESE) 

$19.5 bil 


o 


o 


Office of Educational 
Technology (OET) 

(In Office of the Secretary) 


Natl. Center for Ed. Statistics 
(NCES) 

$250.0m 


Office of Adult & 

Vocational Education 
(OVAE) 

$1.7 bil 

Office of Postsecondary 
Education 
(OPE) 

$2.5 bil 

Office of Innovation & improvement 
(ON) 

$4.0 bil 



Department of 
Education 

$94.6 bil' 
($392m in dedicated 
R&D funding)®** 



• 

• 

U.S. Dept, of 

Other Federal 

Education 

Agency 


o 



NAEP Assessment 

$1 38.3m 

^ Statewide Data Systems 
^ $38.1 m 



Institute of 
Education Sciences 
(lES) 

$593.7m 



Natl. Center for Ed. Research 
(NCER) — 

$ 168.8m 


Natl. Center for Education Evaluation 
and Regional Assistance 
(NCEE) 

$1 14.6m 


National Special Education 
Research Centers 

$49.9m 




National Research Centers 
, $21 0.8m 

(over 5 years) 


Regional Education Labs 

_ $57.4m 


Evaluation of Education 
^ Programs 
$24.8m 


Ed. & Human 
Resources Dir. (EHR) 

$829m 

($555.9m in dedicated R&D 
funding) ^** 


Head Start 
Research & Evaluation 

$20m 


Math and Physical Sciences 
Directorate 
(MPS) 

Computer & Information Science 
& Engineering Directorate 
(CISE) 


Social & Behavioral Sciences Directorate 
(SBE) 


Engineering 

Directorate 

(ENG) 


Game-based Big 
Data Analytics 


intelligent 

Tutors 

Research 


Footnotes on previous page 


Digital Tutor Grand Challenge 
^ $3m 

STEM2Stern Research 
Various Programs 


Math/Science and Related 
Learning Disabilities Program 

$7.0m 

Early Learning Research Program 

$10.3m 


Literacy and Related Learning Disabilities 
$22.7m 


‘Note: The offices represented are not exhaustive within or across agencies, but rather a small sample of research activities. All figures unless otherwise noted 
represent total discretionary allocations, not exclusively funds allocated for education research. Figures do not represent changes in funding due to sequestration. 
Senior staff from cabinet-level agency provided feedback to arrive at represented programs. 

“Funding classified officially as “R&D” is narrowly defined as funding for basic and applied research that excludes activities such as program evaluation, technical 
assistance centers, training, or statistical collection. We include these activities to represent the spectrum of R&D and D activities impacting learning. The $392m in 
Dept, of Ed. funding includes lES R&D funding under that narrow definition, while lES’s budget figure reflects total discretionary appropriations. 


A Sample of Federal Agencies 
Engaged in R&D and Dissemination 
Relevant to Education 


REGIONAL EDUCATIONAL 
LABORATORIES 

THE INSTITUTE OF EDUCATI#N SCIENCES 


Administering Office 

National Center for Eduoation Evaluation and Regional Assistanoe (NCEE) 


Authorizing Legislation 

RL. 107-279, Title I (Eduoation Soienoes Reform Aot of 2002), Part D, Seotion 174 


FY2012 Funding 

Annual Appropriation: $57,426,000 

Total Awarded to Date over 60 Months: $341 .9 million 

Number of Awards: 1 0 

Average Annual Award: $6.8 million 


Purpose and Mission 

The Regional Eduoational Laboratories (RELs) work in partnership with sohool distriots, 
state departments of eduoation, and other stakeholders to use data and researeh to 
improve polioy and praotioe, and henoe aeademie outoomes for students. RELs aot as 
researeh partners, data and analysis eoaehes, and bridge builders for their eonstituent 
states and distriots. They oonduot peer-reviewed researeh and evaluation studies, 
provide teehnieal assistanoe to assist systems in understanding and applying data and 
researeh, and eonneet eduoation researehers with praetitioners through publie forums 
like workshops and webinars. 

To ensure that REL seientifieally based work meets the lES standards for researeh and 
evaluation, all REL fast-response projeet plans, evaluation study plans, and draft and 
final reports are required to pass a rigorous external peer review proeess established 
under NCEE. 


14 The Aspen Institute Education & Society Program 


A 


THE ASPEN INSTITUTE 


EDUCATION & SOCIETY PROGRAM 


FY 2012 Contract Winners 

Each PEL operates under contract to the Institute of Education Sciences at the U.S. 
Department of Education and is overseen by a Governing Board composed of key 
education stakeholders in the region. After a major overhaul of the labs’ design, lES 
hopes to drive deeper improvements in capacity for research -driven practice through 
the latest round of five-year PEL contracts, awarded in 2012. The program prioritizes 
five areas of study for 201 2-2017: 

• Identifying and retaining effective teachers and principals 

• Increasing college readiness, access, and completion 

• Enhancing early childhood education 

• Improving low-achieving schools 

• Adopting and implementing rigorous academic standards and assessments 

The current group of PELs works in partnership with nearly 70 research alliances 
comprised of education practitioners and policymakers at the district and/or state 
levels. Each PEL works with multiple alliances that focus on the region’s high-priority 
education topics, but all maintain a concentration on the five program priorities. 

To learn more about the RELs, visit: http://www.ies.ed.gov/ncee/edlabs/ 


Source: Regional Education Laboratories, Brochure 201 3, http://www.relnei.org/wp-content/uploads/2013/07/REL-10x10-Bitoid-Die-Cut-Brochure_v9.pdt. 
lES National Center for Education Evaluation, http://ies.ed.gov/ncee/edlabs/. 

U.S. Department of Education, http://www2.ed.gov/programs/regionallabs/funding.html. 
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REL Northwest 

(OR, WA,ID, MT,AK) 
Portland, OR 
Education Northwest 
$5.7 million 

educationnorthwest.org/rel-northwest 
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REL Midwest 



(MN,IA,WI,IN,IL,MI,OH) 


REL Northeast and Islands 

Naperville, IL 


(ME, NH, VT, NY, Rl, CT, MA,PR, VI) 

American Institutes for Research 


Waltham, MA 

$9.0 million 


Education Development Center, Inc. 

relmidwest.org 


$7.2 million 



relnei.oro 


REL West 

(CA, NV, UT, AZ) 
San Francisco, CA 
WestEd 
$8.3 million 
relwest.wested.org 


REL Central 

(CO, WY, KS,NE,ND,SD, MO) 
Centennial, CO 
Marzano Research Lab 
$5.8 million 
reicentral.org 


REL Pacific 

(HI, Am. Samoa, Fed. States of Micronesia, Guam, 
R. of Marshall Islands, Palau, N. Mariana Islands) 
Honolulu, HI 
McREL International 
$.4.8 million 
relpacific.mcrel.org 



REL Southwest 

(TX,OK,AR, LA,NM) 
Austin, TX 
SEDL 

$8.0 million 
sedl.org 


REL Southeast 

(MS, AL,GA, SC, NC,FL) 
Tallahassee, FL 
Florida State University 
$7.7 million 
rel-se.fsu.edu 


REL Mid-Atlantic 

(PA, NJ, DE, MD, DC) 
Calverton, MD 
ICF International 
$6.1 million 
relmidatlantic.org 


REL Appalachia 

(KY,TN, WV,VA) 
Alexandria, VA 
CNA 

$5.6 million 
RELAppalachia.org 


lES Regional Educational 
Laboratories 

Annual Appropriation Levels of Five-year 
Contracts Awarded, FY 201 2 


Source: lES National Center for Education Evaluation, http://ies.ed.gov/ncee/edlabs/. 

General Services Administration, Contracts & Acquisitions, www.fbo.gov/index?s=opportunity&mode=form&id=087c8281 60a36b255672d01 dfl 4b1 b2c&tab=core&_cview=1 . 


COMPREHENSIVE CENTERS 
PROGRAM 

U.S.DEPARTMErjTO „U( 


Administering Office 

Office of School Support and Rural Programs 
(Office of Elementary and Secondary Education) 

Authorizing Legislation 

RL. 107-279, Title II (Educational and Technical Assistance Act of 2002), Section 203 


FY2012 Funding 

Annual Appropriation: $ 51 ,1 13,213 
Number of New Awards: 22 
Average Annual Award: $2,370,237 


Evaluation 

In addition to its past five-year evaluations of the program, the Institute for Education 
Sciences is conducting a five-year project to build an ongoing evaluation system for 
the 22 centers, to be based on The World Bank’s ongoing peer evaluation model. 


Purpose and Mission 

The Comprehensive Centers Program awards discretionary grants that support 
22 comprehensive centers to help increase state capacity to assist districts and 
schools in meeting student achievement goals under the Elementary and Secondary 
Education Act (ESEA). By statute, the Department is required to establish at least 
one center in each of the 1 0 geographic regions served by the Department’s regional 
educational laboratories. 
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The 2012 reorganization of the eomprehensive network reorganized the Regional 
Centers, inereased the number of foous-area Content Centers, and reoriented the 
primary foous of teohnieal assistanoe to states only. Therefore, the 15 Regional Centers 
provide serviees primarily to State Eduoation Agenoies (SEAs), while the 7 eontent 
oenters foous on speoifio areas of expertise and may work direotly with SEAs or 
through regional oenters. Grantees are required to develop five-year plans for oarrying 
out authorized aotivities that address state and regional needs. 

15 Regional Comprehensive Centers 

Regional Comprehensive Centers provide frontline teohnieal assistanoe and training 
to states to help them implement ESEA and other related federal sehool improvement 
programs, as well as to help inorease their oapaoity to assist their distriots and sohools. 
Regional Centers are embedded in their regions and responsible for developing strong 
oomprised of eduoation praotitioners and polioymakers at the distriet and/or state 
levels. 

7 Content Centers 

Together, the seven Content Centers oover a speotrum of sohool improvement 
and teohnieal assistanoe areas. Eaoh of them is responsible for providing in-depth 
knowledge, expertise, and analyses in its foeal area to the Regional Centers and the 
states they serve. Content Centers disseminate information about soientifioally-based 
researoh on effeotive praotioe and researohbased produots in their area of speoialty 
and provide expertise that Regional Centers oan use in delivering teohnieal assistanoe 
to states. 


Sources: U.S. Department ot Education, www2.ed.gov/programs/newccp/index.html. 

institute tor Education Sciences. 2011. National Evaluation ot Comprehensive Technicai Assistance Centers, http://ies.ed.gov/ncee/pubs/201 1 4031 /pdf/201 1 4032.pdf. 
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Content Centers 


• Center on Standards and Assessment 
Implementation 

San Francisco, CA 
WestEd 

Partner: CRESST 
$1.4 million 
csai-online.org 

• Center on Great Teachers and Leaders 

Washington, DC 

American Institutes for Research 
Partners: CCSSO, Public Impact 
$2.5 million 
gtlcenter.org 

• Center on School Turnaround 

Sacramento, CA 
WestEd 

Partners: Academic Development Institute, 
Darden/Curry Partnership (UVA), NIRN 
$1.4 million 

centeronschoolturnaround.org 

• Center on College and Career 
Readiness and Success 

Washington, DC 

American Institutes for Research 
Partners: AYPF, the College Board, 

Quill Research Associates, NASDCTEc, 
Forum for Youth Investment 
$1.4 million 
ccrscenter.org 

• Center on Enhancing Early Learning 
Outcomes 

New Brunswick, NJ 
Rutgers University 

Partners: Education Development Center, 

CCSSO 

$1.4 million 

ceelo.org 



• Center on Building State Capacity and 
Productivity 

San Antonio, TX 
Edvance 

Partners: CRPE, Academic Development 
Institute, Edunomics Lab (Georgetown U.) 
$1.4 million 
bscpcenter.org 


• Center on Innovations in Learning 

Philadelphia, PA 
Temple University 

Partner: Academic Development Institute 

$1.4 million 

centeril.org 


Source: U.S. Department of Education, http://www2.ed.gov/programs/newccp/index.html. 
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Texas 

$3.9 million 

\ 


SEDL 



txcc.sedl.org 

> 


Northeast 

CT, MA, ME, NH, NY, Rl, VT 
$3.6 million 
RMC Research 
northeastcompcenter.org 



Mid-Atlantic 

DC,DE,MD,NJ,PA 



$3.0 million 



WestEd 



macc-atwested.org 

J 


Appalachia 
Regional 

KY,TN,VA,WV 
$2.4 million 
ICF International 
arccta.org 


Southeast 

AL, GA, MS, NC, SC 
$4.4 million 
SEDL 

secr.sedl.oro 



Florida and Islands 

FL, PR, VI 
$2.3 million 
ETS 

ets.org/flicc 


Comprehensive Centers 
Network 

As Appropriated in FY 201 2 


NATIONAL EDUCATION RESEARCH 
AND DEVELOPMENT CENTERS 


Since 2004, the National Center for Education Research (NCER) within the Institute 
of Education Sciences has funded nineteen National Education Research and 
Development Centers. The mission of the R&D Centers is to contribute to the production 
and dissemination of rigorous evidence and products that provide practical solutions 
to important education problems in the United States. The R&D Centers achieve this 
mission by developing, testing, and disseminating new approaches to improve teaching 
and learning, and ultimately, student achievement. Each of the Centers conducts a 
focused program of education research in its topic area and works cooperatively with 
NCER to conduct supplemental research and national direction in that area. 


Currently Operating with NCER Funding (by original grant year) 


Center for Analysis of Longitudinal Data in 
Education Research (CALDER) 

American Institutes for Research 

$1 0.0 million (201 2) and $1 2.0 million (2006) 

Center for the Study of Adult Literacy 
(CSAL): Developing Instructional 
Approaches Suited to the Cognitive and 
Motivational Needs for Struggling Adults 

Georgia State University 
$10.0 million (2012) 

Center for Analysis of Postsecondary 
Education and Employment (CAPSEE) 

Columbia University, Teachers College 
$10.0 million (2011) 

Center on Cognition and Mathematics 
Instruction 

WestEd (Redwood City, CA) 

$10.0 million (2010) 


Center on Scaling Up Effective Schools 

Vanderbilt University 
$13.6 million (2010) 

Center for Research on Rural Education 

University of Nebraska-Linooln 
$10.0 million (2009) 

Center for Teacher Effectiveness: Validating 
Measures of Effective Math Teaching 

President and Fellows of Harvard College GSE 
$10.0 million (2009) 

Center on Cognition and Science 
Instruction 

21 st Century Partnership for STEM Education 
$10.0 million (2008) 

Center on Instructional Technology: Center 
for Advanced Technology in Schools 

UCLA 

$9.8 million (2008) 
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Center on Instructional Technology: 

Possible Worlds 

Education Development Center, Inc. 

$9.2 million (2008) 

Originally Funded under NCER, Now 

Center for Postsecondary Research 

Columbia University, Teachers College 
$9.8 million (2006) 

Center on the Gifted and Talented 

University of Connecticut 
$8.7 million (2006) 

Center for Research on Evaluation, 
Standards, and Student Testing (CRESST) 

UCLA 

$12.5 million (2005) 

No Longer Operating 

Center for Performance Incentives (NCPI) 

Vanderbilt University 
$10.8 million (2006) 

Center for Research on Early Childhood 
Education 

University of Virginia 
$11.0 million (2006) 


Operating Under External Funding 

Center for Data-Driven Reform in Education 

Johns Hopkins University 
$13.6 million (2004) 

Center on School Choice 

Vanderbilt University 
$13.2 million (2004) 


Center for Research on the Educational 
Achievement and Teaching of English 
Language Learners (CREATE) 

University of Houston 
$9.9 million (2005) 

Center on Rural Education Support 

University of North Carolina, Chapel Hill 
$17.2 million (2004) 


*R&D Center Awards are rounded to the nearest $100,000. All R&D Center awards are for five years. However, several centers received additional time to complete their 
activities through no-cost extensions. The Center for Analysis of Longitudinal Data in Education Research received two consecutive five-year awards. Centers no longer 
receiving lES funding may still be conducting research with other funding sources. 

Source: lES staff; NCER Research and Development Centers, ies.ed.gov/ncer/randd/. 
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SPECIAL EDUCATION RESEARCH AND 
DEVELOPMENT CENTERS 


The Special Education Research and Development Centers are funded through IBS’s 
National Center for Special Education Research (NCSER)*. The mission of the R&D 
Centers is to contribute to the production and dissemination of rigorous evidence and 
products that provide practical solutions to important education problems in the United 
States. The R&D Centers achieve this mission by developing, testing, and disseminating 
new approaches to improve teaching and learning, and ultimately, student achievement. 
The Centers focus their efforts by topic area and provide supplemental research and 
national direction on that topic in partnership with NCSER. 


Improving Reading and Mathematics 
Outcomes for Students with Learning 
Disabilities 

Vanderbilt University 
$10 million (2013) 

Center on Reading Instruction for 
Deaf and Hard of Hearing Students 

Georgia State University Research 
Foundation, Inc. 

$10.0 million (2012) 

Center on Secondary Education 
for Students with Autism Spectrum 
Disorders (CSESA) 

University of North Carolina, Chapel Hill 
$10.0 million (2012) 

Center on Assessment and 
Accountability for Special Education 

University of Oregon 
$11.7 million (2011) 


Center on Improving Mathematics 
Instruction for Students with 
Mathematics Difficulties 

University of Delaware 
$9.9 million (2010) 

Center for Response to Intervention 
in Early Childhood 

University of Kansas 
$10.0 million (2008) 

Center on Serious Behavior Disorders 
at the Secondary Level 

Lehigh University 
$10.4 million (2008) 


*R&D Center Awards are rounded to the nearest $100,000. All R&D Center awards are for five years. However, several centers received additional time to complete their 
activities through no-cost extensions. The Improving Reading and Mathematics Outcomes for Students with Learning Disabilities Center was funded through NCSER’s 
Accelerating the Academic Achievement of Students with Learning Disabilities Research Initiative. 

Source: NCSER Research and Development Centers (http://ies.ed.gov/ncser/RandD/), Accelerating the Academic Achievement of Students with Learning Disabilities Research 
Initiative (http://ies.ed. gov/ncser/projects/program.asp?ProglD=78), and lES staff. 
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NATIONAL RESEARCH CENTERS FOR 
CAREER AND TECHNICAL EDUCATION 


In 2006, Perkins IV legislation authorized the ereation of a National Researoh Center 
for Career and Teohnioal Eduoation (NRCCTE) that inoluded the funotions of researoh, 
teohnioal assistanoe, professional development, and dissemination. Between 2007 
and 2012, the NRCCTE had been led by the University of Louisville in partnership with 
a oonsortium of institutions and organizations involved in CTE today. The grant was 
oonverted to a oontraot in 201 2 and opened to for-profit oompetitors. RTI International 
now has a three-year oontraot to open a similarly missioned oenter while NRCCTE 
eontinues to operate largely from state teohnioal assistanoe funding. Given their main 
aotivities, the two organizations below resemble oomprehensive oenters more than 
formal R&D oenters. 


National Research Center for Career 
and Technical Education (NRCCTE) 

University of Louisville 
$19.9 million (2007-2012) 


National Center for Innovation in Career 
and Technical Education (NCICTE) 

RTI International 
$9.2 million (2012-2015) 
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Investing in Innovation 

"l/l/e need innovations that innprove outcomes, and that can and will be 
used to serve hundreds of thousands of teachers and millions of students." 

-Arne Duncan 

Office of Innovation and Improvement 
U.S. Department of Education 


Investing in Innovation (i3) Overview Three Types of i3 Grants 


The i3 program, established in 2009 under the American 
Recovery and Reinvestment Act (ARRA), is managed by 
the Department of Education’s Office of Innovation and 
Improvement. The program supports the development 
of ambitious new efforts and the scaling of effective 
practices that improve K-12 student achievement, 
particularly among high-need students. Through a 
competitive grant competition, i3 provides funding to 
either local educational agencies (LEAs) or nonprofit 
organizations in partnership with LEAs or schools with 
proven records of improving student achievement. 
These grants can be for three to five years. Grantees’ 
projects focus on a range of issues in urban, suburban 
and rural settings, including: 

• Improving teacher and principal effectiveness 

• Improving student achievement in STEM 

• Improving family and parental engagement 

• Improving low-performing schools 

• Improving outcomes for English learners and 
students with disabilities 

• Supporting the effective use of technology 

Two features of the i3 program are particularly notable; 

1 The amount of funding for which grantees are 
eligible depends on the previously demonstrated 
level of effectiveness of their program. 

2 All grantees are required — and receive extensive 
support — ^to conduct and publicly share a rigorous 
evaluation that assesses the impact of the i3- 
funded program. 


Development. Smaller grants to support high-potential but 
relatively untested projects; grants range from $3-$5 million. 
There are currently 59 Development grants. 


The Exploratorium Institute for Inquiry’s project 
Integrating English Language Deveiopment and 
Science: A Professional Development Approach provides 
professional development to teachers at an elementary school 
in California's Sonoma Valley School District to improve 
English learners’ language skills through science. In addition to 
supporting students’ science and language learning, students’ 
confidence in their English language ability has also improved. 


Validation. Grants for programs with moderate evidence of 
effectiveness to expand their capacity; grants range from 
$15-$30 million. There are currently 28 Validation grants. 


University of Missouri’s eMINTS (enhancing Missouri’s 
Instructional Networked Teaching Strategies) project is 
focused on changing teacher practices to improve student 
learning and build students’ 21st century skills. Teachers are 
developing engaging, standards-aligned, assessment-informed 
classroom-level practices that employ technology. Data to date 
suggests that classroom instruction is being transformed and 
student learning in math and arts has improved. 


Scale-up. Large grants for proven, broad-reaching 
programs, with strong evidence and track records of 
success; grants range from $25-$50 million. There are 
currently 5 Scale-up grants. 


The Ohio State University’s Reading Recovery project 
prepares over 3,500 new Reading Recovery teachers in 
low-performing and rural schools, and schools with a high 
concentration of English learners. These teachers are working 
with 90,000+ students one-on-one, and 400,000+ students 
in small-group instruction. Initial findings indicate that Reading 
Recovery students are learning at a rate between 1 20% and 
1 66% faster than comparable students. 
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i3 By the Numbers 

• Grantees 

» 92 grantees total 
» 49 from FY10 
» 23 from FY1 1 
» 20 from FY1 2 
» 59 Development grants 
» 28 Validation grants 
» 5 Soale-up grants 

• Doliars 

» Nearly $1 billion in federal funds Invested from 
FY10 through m2 

» Over $182 million in matohing private funds to 
support I3 projects 

• Students 

» Over 1 million students (total) served in 201 1 
and 2012 

» Building evidence of what works, for whom and 
in what context 

» 76 projects are on track to meet the 

Institute of Education Sciences What Works 
Clearinghouse’s (WWC) evidence standards 

NOTE: In November 201 3, The Department announced its 
2013 25 Highest-Rated Applications for the I3 program. 
These applicants have until December 1 1 , 201 3 to secure 
50% of their private-sector match funding. Applicants 
that secure funds become i3 grantees. Early in 2014, 
updated program information will be available, including 
number of total grantees, funding amounts, and areas of 
project focus. 

Building a Portfolio of Solutions 

With 92 current projects expending over $1 billion in 
federal and private funds, the i3 program represents 
a large portfolio of solutions to address some of the 
most important issues facing school leaders, teachers, 
students and families today. The program invests in 
innovation in two primary ways. First, the program 
supports promising projects that are new or untested. 
These projects, in turn, build evidence of which practices 
work, for whom, and in what oontexts, while enabling 
the broader education field to learn about their impact. 
As promising practices are tested and evaluated, 
solutions that have proven impact to address some of 
the most pervasive and vexing problems in education 
begin to emerge. 


The program also invests in innovation by supporting 
the scaling of effeotive solutions or interventions — 
whether previously supported by Department programs 
like i3 or otherwise. Supporting the expansion of proven 
interventions to serve large populations of high-need 
students across a range of contexts has the potential to 
dramatically enhance student performance nationwide. 
In addition, such efforts could ultimately displace 
widespread but less effective interventions. 

Each i3 project, with the support of the Department of 
Eduoation’s staff and a range of external experts, has 
various tools to monitor their progress. This enables 
them to ensure that they are implementing their project 
with fidelity and on track to meet important project 
goals and milestones. Additionally, most projects have a 
“treatment” and a “control” group so that the projeot can 
be rigorously evaluated to understand the impaot of the 
intervention or program. To date, 76 of the 92 projects 
are on track to meet the What Works Clearinghouse’s 
evidenoe standards (note that this may shift as the 
projects and their evaluations evolve over time). 
Generating this kind of new information and enabling 
broad dissemination about i3-tunded interventions is a 
key element of the program’s commitment to knowledge 
development and sharing. 

For more information, visit www.ed.gov/programs/ 
innovation. Follow us on Twitter @ED_OII 
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Why We Need a DARPAfor Education (ARPA-ED) 

And in the same way that we invested in the science and research that 
led to the breakthroughs like the Internet, I'm calling for investments in 
educational technology that will help create digital tutors that are as 
effective as personal tutors, and educational software that's as compelling 
as the best video game. 

- President Barack Obama announcing ARPA-ED proposal, March 8,2011 

U.S. Department of Education WORKING DRAFT 


Background 

Effectively educating all Americans will be critical to 
maintaining our nation’s economic competitiveness and 
our citizens’ quality of life. Aohieving our aspirations — 
ensuring every student is oollege- and oareer-ready and 
our nation is leading the world in college attainment — 
will require dramatic increases in student achievement, 
at all levels of the educational system, in a relatively 
short period of time. 

Technology has transformed almost every aspect of 
society and could have a similar impact on learning: the 
ever increasing access to technology and subsequent 
access to information and content that technology 
affords, the ability to personalize for users, and the 
creativity and productivity technology has brought to so 
many other experiences. 

Overthe past SOyears, DARPAhas had atransformational 
impact on America’s military and our nation’s economy 
by making investments in high-risk, high-return research 
and development (R&D) and gathering the best minds in 
key fields (both at DARPA as program managers and as 
funded researchers). This has led to advances like the 
Internet, GPS, and stealth technology. 

The nation needs a similar capability to accelerate 
transformative innovation in education and learning, 
particularly learning technology — taking advantage of 
advances in information and communications technology 
(e.g., low-cost networked devices, cloud computing, 
machine learning, speech recognition technology) and 
building on recent advances in the science of learning 
(e.g., neuroscience, cognitive psychology). 


That’s why the President’s recent budget proposals 
have called for the creation of an Advanced Research 
Projects Agency — Education (ARPA-ED) to aggressively 
pursue technological breakthroughs that transform 
educational technology and empower learning and 
teaching. 

Rationale 

The substantial potential of technology to assist learning 
is unrealized: research has shown that the education 
sector is among the least-changed by the information 
technology innovations that have revolutionized other 
parts of the economy^ . 

This limited investment in R&D in part means that 
promising research discoveries often are not translated 
into innovations that can make a difference for students 
and teachers. 

One reason is that research and development (R&D) 
accounts for only 0.2 percent of total national K-12 
expenditures^. In contrast, knowledge-intensive sectors 
of the economy invest 10-20 percent of sales in R&D, 
and even mature industries devote 2 percent of sales 
to R&DT 

Advances empower both students and teachers. Too 
often, teachers are expected to diagnose learning 
needs for a classroom full of students and differentiate 
instruction with little more than a textbook and a 
blackboard. ARPA-ED would provide teachers with tools 
that make them far more effective like those that doctors, 
engineers, and other professionals take for granted. 
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IT-based solutions have the potential to scale rapidly, 
since the marginal cost of making software and digital 
content available to more students is low. 

American leadership in learning technologies can also 
create a new growth industry. Other nations have 
recognized the importance of using technology to 
empower teaching and learning. South Korea, already 
one of the world’s highest-performing school systems, 
recently announced that by 2015 the entire K-12 
curriculum will be digital-based'^. Uruguay has rolled out 
a “one laptop per child’’ initiative and is building on top 
of that infrastructure®. Investments by DARPA laid the 
groundwork for the Internet and GPS, both multi-billion 
dollar industries employing thousands of Americans, 
and in much the same way ARPA-ED could catalyze a 
new domestic and export industry. 

Examples of where promising research 
could lead to impact 

ARPA-ED would help address that underinvestment 
and accelerate the impact of technology on learning. It 
will complement and build upon innovative work being 
done across the public sector and the private sector, 
and could catalyze projects like: 

• Digital tutors that are as effective as personal 
tutors, moving the average student from the 50* 
percentile to the 98* percentile®'^'® 

• Instructional designs and tools that take advantage 
of technology and learning sciences research to 
dramatically improve teacher effectiveness (greatly 
improving student outcomes or reducing the time 
to learn content)®'^® 

• Courses that improve the more students use 
them, based on analyzing user data to identify 
patterns, tailor results to users’ preferences, and 
provide a more individualized experience (much 
like consumer companies like Netflix or Pandora 
already do)" 

• Discovery games and software that not only 
teach students advanced topics in core academic 
subjects but also contribute to scientific discovery — 
and, in so doing, make students’ classroom 
experience authentic and relevant again''^’''®''''' 

• Low-cost networked devices that use advances 
in speech recognition and text-to-speech to teach 
early literacy and reduce the large gap in the size 
and sophistication in vocabulary size between 
young children in rich and poor households'®'^®'"'"'® 


Demand Strategies for ARPA-ED 

To achieve success, ARPA-ED not only must invent 
technologies that have the potential to improve 
student achievement but also help transition and 
implement successful projecfs that make demonstrable 
improvements in student achievement, particularly for 
the highest-need students. 

Unlike DARPA, after which ARPA-ED is modeled, and 
its relationship with the U.S. military, there is no unitary, 
large purchaser of the tools and products that ARPA- 
ED will develop. Thus, there is no single adoption and 
implementation pathway to scale. Successful ARPA-ED 
projects will need to explore a range of strategies for 
follow-on funding and adoption. 

The Department has identified multiple strategies 
for supporting the transition and implementation 
of ARPA-ED projects beyond the period of federal 
funding. After ARPA-ED starts up, one of the critical 
early activities of the initial staff will be to formalize 
these strategies and identify others. 

Demand side strategies. Different industries have 
explored a range of strategies to create intelligent 
demand and increase the ability of adopters to guide 
markets. In particular, the global health community has 
used a tool called Advance Market Commitments to 
create purchase orders for vaccines that do not exist, 
in order to signal a ready-made market that providers 
could address. Another approach, purchasing 
consortia, has been used effectively in education: 
New Hampshire, Vermont, Rhode Island, and Maine 
formed a consortium called the New England Common 
Assessment Program to buy testing materials together, 
getting a higher quality product at a lower cost. 

Partnerships with eariy adopters. Several 
organizations with substantial purchasing power 
have expressed interest in being early adopters 
of advanced education technologies. In a recent 
Presidential Study Directive on Military Families, the 
Department of Defense committed to making the 
Department of Defense Education Authority a leader 
in the use of advanced learning technologiesIQ. In 
the civilian sector, the League of Innovative Schools, 
a project of Digital Promise, is convening districts 
and schools that are interested in being test beds 
for innovation in education technology, helping 
validate the most advanced technology and bring it 
into practice. 
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Commercialization support. The Advanced Research 
Projects Agency-Energy (ARPA-E), within the U.S. 
Department of Energy, also does not have a single 
purchaser and must confront a diffuse market for 
bringing its products into use. ARPA-E considers 
prospects for commercialization as a factor in selection 
of its performers and then aggressively supports its 
funded projects through an in-house commercialization 
group, helping performers identify potential customers 
and develop a go-to-market strategy while they are 
advancing their technology. 

In addition to structural supports to help ARPA-ED 
projects reach students and teachers, there are other 
reasons to believe that if ARPA-ED can achieve its 
ambitions, adoption will follow. 

Response to outsize performance. Products that 
emerge to address specific needs in education — such 
as the tools developed by Wireless Generation or the 
instructional videos provided by Khan Academy — have 
been able to expand substantially in reasonable time 
periods. Because ARPA-ED is focused on specific 
projects that will create tools and capabilities of practice 
value and substantial impact, these examples provide 
strong reason to believe that the market will respond to 
outsized performance. 


Existing momentum. ARPA-ED will begin its work 
with the benefit of current efforts such as the state- 
led work on common standards that has put over 
40 states on track to share common standards in 
mathematics and English language arts, as well as 
aligned assessments. The efforts of the National 
Governors Association and the Council of Chief State 
School Officers to establish nationwide yardsticks 
support education research and development 
generally and will particularly assist ARPA-ED by 
enabling the organization to develop solutions relevant 
for large numbers of studenfs. 

While commercialization and scale will be substantial 
challenges to the success of ARPA-ED, the strategies 
listed above and others that will emerge can help 
address the challenge. Success in that effort will be 
beneticial not only to ARPA-ED, but can also increase 
informed and intelligent demand across the entire 
market for educational content, materials, and services. 


' http://www.esa.doc.gov/sites/default/files/reports/clocuments/dig_econ_2003.pdf 
2 http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-stemed-report.pdf 
^ http://www.booz.com/media/uploads/lnnovation_1 000-2009.pdf 
* http://www.edweek.org/ew/articles/2011/08/10/37korea.h30.html 

^ http://www.olpcnews.com/countries/uruguay/uruguays_plan_ceibal_unveils_its_plans_for_2011.html 
® http://www.comp.dit.ie/dgordon/Courses/ILT/ILT0004/TheTwoSigmaProblem.pdf 
r http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA422304 
“ http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA542215 

5 http://www.medicine.virginia.edu/education/medical-students/UME/faculty-development-nxgen-teach-the-teachers/journal-club-articles/JournalCiubJLIN7.pdf 
http://oli.web.cmu.edu/openlearning/files/theinitiative/publications/jime-2008-14.pdf 
" http://www.darpa.mil/Our_Work/l20/Programs/ENGAGE.aspx 
'2 http://getdown.org/ben/projects/anatomy-pro-am/ 

'3 http://www.cs.washington.edu/homes/zoran/NSMBfoldit-201 1 .pdf 
''' http://www.nap. edu/catalog.php?record_id=1 3078 
http://eric.ed.gov/ERICWebPortal/detail?accno=EJ54481 3 

http://www.strategiesforchildren.org/eea/6research_summaries/05_MeaningfulDifferences.pdf 
'r http://www.jstart.org/site/DocServer/America_s_Eariy_Childhood_Literacy_Gap. pdf?doclD=3923 
http://www.sedl.org/reading/framework/research.html 

'3 psD-9, Strengthening our Military Families. “DOD commits to making DOD Education Activity (DODEA) schools a leader in the use of advanced learning 
technologies that have the potential to significantly improve student performance. DOD will invest in research, development and demonstration projects that 
involve DODEA schools, building on relevant research by agencies such as the Defense Advanced Research Projects Agency and the Office of Naval Research. 
Examples of advanced learning technologies include software that is as effective as a personal tutor, engaging and compelling games for learning, student- 
authored and sharable simulations, and online courses that improve the more students use them.” http://www.defense.gov/home/features/2011/0111_ 
initiative/strengthening_our_militaryjanuary_201 1 .pdf 
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Strengthening the Education Sciences 
Reform Act: Fostering Innovation and Efficiency 
through Mechanism Experiments 

Timothy Knowles and Jens Ludwig^ 

University of Chicago 


The US Department of Education founded the Institute 
of Education Sciences (lES) with a primary focus on 
running randomized experiments; its annual budget in 
recent years has been a little under $600 million^ What 
have we gotten from this investment? In testimony to 
the House Ways and Means Subcommittee on Human 
Resources in July 2013, Jon Baron, President of the 
Coalition for Evidence-Based Policy, noted: “Of the 
90 interventions evaluated in randomized trials by 
the Institute of Education Sciences (lES) since 2002, 
approximately 90% were found to have weak or no 
positive effects” (Baron, 2013, Coalition for Evidence- 
Based Policy, 2013). 

Given this track record, what should Congress do as it 
considers re-authorization of the law that initially created 
lES back in 2002 (the Education Sciences Reform 
Act)? One possibility is to give up on the lES focus on 
randomized experiments. Another would be to give up 
on lES altogether. We think either would be a mistake. 
The initial motivation for lES to focus on experiments 
comes from the difficulty of non-experimental methods 
and data in isolating causal relationships between 
student learning and different educational policies 
and practices. While non-experimental research is 
important, its limitations have not gone away. 

A more constructive response would to be figure out 
ways of getting more bang-per-buck from the money 
lES spends on randomized experiments. Specifically we 
think there would be great value in having lES take a 
broader view of the sort of experiments it will support 
to include more “mechanism experiments.” Unlike 
experiments that test a policy or program (what we call 
standard “policy evaluations”), mechanism experiments 
test the key mechanisms-of-action through which 
policies or programs might affect student learning. 
By undertaking more mechanism experiments, lES 
could increase the chances that the full-scale policy 
experiments that it does fund will show beneficial 


impacts and — overtime — improve the quality of teaching 
and learning in the American school house and beyond. 

Mechanism Experiments 

What is a mechanism experiment, and how does it 
differ from, and relate to, a large-scale randomized 
trial evaluating a new policy or program? A mechanism 
experiment does not test a policy, rather it directly 
tests a causal mechanism that could underlie a policy. 
Sometimes such experiments can generate much more 
information per dollar spent about a policy than directly 
testing the policy itself. 

“A mechanism 
experiment tests a 
causal mechanism 
that could underlie a 
policy instead of the 
policy itself.” 

Consider for example the question of whether 
converting public high schools to “small schools” can 
improve student achievement. Why might that be the 
case? Or put differently, what are the mechanisms 
through which a small school might promote student 
engagement with school? One possibility is that 
students might form stronger bonds with school staff 
in smaller schools. The logic model for this policy is: 
small schools — > bonding — > student learning. 

How would we test whether small schools are actually 
as helpful as some people hope? The most obvious way 
to do this would be to carry out a policy evaluation — 
randomly assign some schools but not others to 
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receive the small school reform then compare the 
effects on student learning in the two types of school 
environments. While that sort of policy evaluation could 
be useful, it would typically be incredibly costly, since 
the costs of the intervention per school are significant 
and the number of schools that would be required to 
allow for statistical measurement of program impacts 
would be large. 

But suppose that educators and policymakers already 
felt confident that smaller schools promote student 
bonding, but were unsure about whether greater 
student bonding with teachers would translate into 
improved student learning. (That is, policymakers felt 
they understood the small schools — > bonding link but 
did not understand the bonding — > student learning 
link). The large-scale policy evaluation that randomizes 
some schools to become “small schools” and others to 
continue operating under the status quo model might 
not be the most cost-effective experiment to test the 
value of smaller schools. The reason is that the policy 
evaluation expends resources testing both links of the 
chain in the logic model, and doesn’t capitalize on the 
fact that we think we already know something about 
one of them. 

Consider instead, this alternative. Identify a set of 
schools, and within those schools assign some teachers 
but not others the opportunity to earn overtime pay to 
work after school and on weekends with small, randomly 
selected groups of students, in an effort to promote 
faculty-student bonding. If this mechanism experiment 
found no effect of improved faculty-student bonding 
on student learning, would that change the thinking of 
policymakers about the value of small schools? Would 
we even need to run a full-scale policy evaluation? 

To extend this example, suppose that the results of 
the mechanism experiment were positive — increased 
student-faculty bonding improved student learning. 
If policymakers were extremely confident in their 
understanding of the small schools — > bonding link, 
and if carrying out a full-scale policy evaluation were 
costly enough, they might choose to adopt the policy 
straight away. If policymakers were unsure about 
the small schools — > bonding relationship, or if an 
evaluation of this policy were not overly expensive 
compared to the cost of carrying out the actual policy 
itself at scale, policymakers might wish to follow the 
successful mechanism experiment with a full-blown 
policy evaluation. But the mechanism experiment has 
now served as a useful filter to increase the chances that 


the policy evaluation will generate positive results. That 
is, mechanism experiments can increase the efficiency 
of the IBS policy-evaluation portfolio by helping weed 
out “bad bets” in advance. 

Another type of mechanism experiment that can serve 
this function is one that delivers an intervention that 
mostly looks like a policy that might be adopted, but is 
administered at a much higher dose than would ever be 
done at large scale. The value of this sort of mechanism 
experiment relative to a policy evaluation that delivers 
a more realistic dosage is that the former (but not the 
latter) can help shut down an entire line of inquiry. 
Consider a conditional cash transfer intervention (CCT) 
that pays parents or students some amount contingent 
on how the student does in school. If we carried out a 
policy evaluation of the sort of CCT payments that might 
be adopted at scale did not find statistically significant 
results, there would always be the question of whether 
a slightly more generous payout structure would 
generate more pronounced results. If we started with a 
mechanism experiment that applied a massive dosage 
but still found no effect, we could avoid the expense of 
carrying out subsequent CCT policy evaluations. 

Implications for lES 

Our argument here runs counter to the critique that 
was leveled by many economists against the large- 
scale social experiments carried out by the federal 
government in the 1 970s and 1 980s for “not necessarily 
testpng] real policy options” (Harris, 1985, p. 161). We 
think this represents an overly narrow view about what 
sorts of randomized experiments can have value for 
education policy. Mechanism experiments can also play 
a crucial role. 

We note that IBS does currently support some “tier 
1 ” and “tier 2” experiments that might be considered 
mechanism experiments. Some of these experiments 
test interventions applied at some “dosage level” that 
is much higher than what policymakers might actually 
implement at large scale. 

“Mechanism 
experiments can 
heip increase the 
sociai good per iES 
doiiar spent ” 
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Broadening our view of what sorts of experiments are 
valuable for education policy to include mechanism 
experiments can help increase the social good per 
dollar spent that derives from the lES research budget. 
This occurs both directly, in some cases by helping 
rule out candidate policies at lower cost than can be 
done with a large-scale policy evaluation, and indirectly, 
by helping winnow out bad bets for experiments to 
increase the odds of success for the policy evaluations 
that are carried out. 


Put differently, mechanism experiments should best 
be thought of as complements to, not substitutes for, 
policy evaluations. We would not argue that the optimal 
portfolio of lES experiments should consist exclusively 
of mechanism experiments. Rather, increasing the 
share of the lES research portfolio devoted to this type 
of research activity might increase the odds that Jon 
Baron can report out more optimistic statistics in 10 
years at the next re-authorization of lES. 
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The Education Sciences Reform Act (ESRA) of 2002 
launched a period of greafer rigor in education research, 
especially for “what works” studies examining whether 
programs demonstrate impact. The challenge ahead 
is strengthening the relevance of education research, 
without sacrificing the rigor. 

Below I discuss six issues pertinent to the reauthorization 
of ESRA, focusing on ways to improve the use and 
usefulness of education research. 

1 Relevant to whom? Calls to make research more 
relevant have been ringing for decades, but doing 
so proves challenging. In part, this is because the 
criteria forjudging relevance are not clearly defined. 
This includes lack of consensus as to whom research 
should be relevant. 

ESRA reflects this challenge. The Act includes a 
long list of would-be research users: teachers, 
administrators, librarians, other practitioners, 
parents, policymakers, voluntary organizations, 
professional associations, the media, the general 
public, and researchers themselves. This list makes 
a great deal of sense given that federally funded 
research should serve the public interest. But, this 
far-reaching mandate also presents a big lift. After 
all, different stakeholders look to research to resolve 
varying questions and concerns, and satisfying 
everyone equally is not an easy task. 

In other writing, I have argued for greater attention 
to the research needs of state and local education 
agencies. No Child Left Behind, Race to the Top, and 
the Investing in Innovation (i3) program have sought 
to promote the use of research in state and local 
decisions about curricula, professional development, 
turning around low-performing schools, evaluating 
teachers, and improving student test scores. There 
is still a sizable gap, however, between the research 
available to districts and their needs. Reauthorization 
of ESRA could signal a stronger focus on identifying 
and addressing the research needs of state and 
district decision-makers. 


2 Relevant for what questions? Congress’ 
reauthorization of ESRA provides a useful juncture to 
pause and reconsider the goals for federally funded 
research. Over the past decade, federal efforts 
have increasingly focused on building knowledge of 
what works. That agenda has provided a practical 
focus for the education research community. For 
district administrators, however, it has not provided 
enough relevant information to make decisions 
locally. Knowing what has worked elsewhere is 
only somewhat useful. What districts need to know 
is whether a program, curriculum, or professional 
development strategy will be right for their local 
context and, if so, how they should implement 
it. This suggests broadening the “what works” 
agenda to build evidence of what works for whom 
and under what conditions. It also entails better 
addressing districts’ implementation concerns. 
Researchers often study implementation in terms of 
dosage (how much of the program was delivered) 
and fidelity (how closely the services delivered 
match the original model). This is a more limited 
look into implementation than what district leaders 
and administrators actually need. They require 
knowledge of how to align financing, staffing, and 
training — among other things — to achieve effects. 

There are reasons to believe that an even broader 
research agenda would be fruitful. According 
to the Coalition for Evidence-Based Policy, the 
Institute for Education Sciences (IBS) has funded 
90 interventions evaluated by randomized control 
trials (RCTs) since 2002. Approximately 90 percent 
of them found weak or no positive effects.^ Focusing 
narrowly on a “what works” agenda is an exercise 
in frustration unless it is balanced by other types 
of research that generate new ideas and deeper 
knowledge of the persistent challenges in education. 
Others also worry that the “what works” agenda 
encourages districts to chase silver bullets, and that 
simply adopting a new program shown to work in 
other contexts will not improve their systems. They 
can’t simply “program” their way out of problems, 
and research needs to help them understand how 
to build stronger systems. 
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3 Build Research-Practice Connections. In order 
to improve relevance, researchers must have ways 
to learn about the information needs of education 
decision-makers. Researchers, poiicymakers, and 
practitioners work in separate worlds with different 
incentives, goals, language, and rewards. They also 
work at very different paces. If work continues in 
separate spheres with little infrastructure connecting 
them, the ability to make research more useful and 
to get it used will remain stunted. 

Researchers and would-be research users need 
opportunities to interact at the beginning when 
research agendas are formulated. Considering state 
and district needs only after studies are completed 
and findings are ready to be distributed is too late. 
Interaction on the front-end enables researchers 
to make informed choices in study designs, 
measurement plans, and sampling choices to 
address decision-makers’ questions. 

Research-practice partnerships have emerged as 
one promising strategy for researchers to collaborate 
with districts and states to define research agendas 
jointly.^ Partnerships are popping up in cities across the 
country — Baltimore, New York, San Diego, Kansas 
City, Los Angeles, Houston, and New Orleans. At 
the federal level, lES began supporting partnerships 
through a new RFA; they are also funding regional 
research alliances through contracts for the Regional 
Education Labs.^ These partnerships depart from 
the typical way researchers and districts work 
together through one-off projects or consultations. 
Instead, through these partnerships, researchers 
and districts engage in long-term collaborations that 
are designed to help practitioners address more 
complex problems over time. 

In addition to supporting partnerships, a crucial 
role for the federal government is to connect the 
lessons learned across districts and states and to 
disseminate that information broadly and to the 
right stakeholders. Successful partnerships — like 
marriages — are not made overnight. They confront 
the challenges of developing research agendas 
that meet multiple stakeholders’ needs, navigating 
the different timeframes between research and 
practice, maintaining trust even when research 
findings can damage districts’ public images, 
and preserving collaboration as district leadership 
changes. Some partnerships are developing 
smart strategies to address these challenges, but 
there are few mechanisms to aggregate and share 
these strategies. 


Progress rests on inoentivizing and rewarding 
researeh-practioe connections. The current system 
rewards researchers for publishing in aoademic 
journals. While the peer review process helps to 
ensure the scientific quality of research, it does little 
to address the usefulness of researoh for policy and 
practice. Incentive systems should encourage the 
strongest researchers to focus on the persistent 
problems of polioy and praotioe. Moreover, if future 
generations of soholars are to be more apt than 
their predecessors at conducting relevant research 
and communicating it clearly, they will need better 
training than is currently available. They will need to 
develop the skills for collaborating with policymakers 
and practitioners in designing research, writing for 
them, and helping them understand what existing 
research suggests for improving their work. 

4 Strengthen rigor for different types of research. 

lES has greatly improved the field’s capacity to 
oonduot randomized oontrolled trials, but scant 
attention is focused on improving the rigor of 
other types of researoh. Relevant research starts 
by asking the right questions — when the question 
oalls for assessing the impaot of a program, RCTs 
are the best available research design. But when 
other questions arise — as they always do — RCTs 
are not necessarily suited to addressing them. For 
example, states and distriots aoross the country 
are designing and implementing new teaoher 
evaluation systems. They need researoh to help 
them with strong measures of teaching, ways to 
oombine different measures into indices, and ways 
to implement teaoher evaluations in personnel and 
professional development decisions. lES provided 
the national leadership to develop standards for rigor 
in addressing “what works” questions — standards 
that have influenced fields outside of education. It 
oan build on and expand that success to strengthen 
rigor for other types of education research. 

5 Rethink dissemination and promote 
engagement. Producing relevant research only 
gets us part way down the road to research use. 
Dissemination aotivities get us a bit further, but 
those too are insuffioient to promote the uptake 
of researoh. That is because using research is not 
simply a matter of transferring research “facts” from 
researohers to decision-makers, who then apply 
them. Using research involves busy policymakers 
deciphering what research means for their local 
context and determining its implications given their 
goals, resources, and the political ramifications of 
any decision. To do that, they need opportunities 
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to engage with researchers to discuss research 
implications for their local circumstances and to 
translate it into concrete actions. 

Moreover, we have overly relied on written 
products and websites as dissemination tools and 
underestimated the value of strong networks and 
trusted relationships. State and local agencies often 
look to their peers working in similar contexts and 
to key intermediaries — professional associations, 
technical assistance providers, consultants — 
as sources of research and for judgments of 
its credibility. Despite the importance of these 
relationships, the research community rarely 
leverages them to enhance research dissemination 
and use. Future efforts should try to identify these 
trusted brokers and collaborate with them to 
ensure that the advice they provide is informed 
by research and that they are strong conduits for 
communicating research. 

6 Maintain iES’ roie as an honest broker. The 

independence of federally-funded education 
research has never been more important. The 
marketplace of education research is becoming 
ever more complex as Foundations, vendors, 
advocacy groups, and think tanks promote research 


to support specific reform agendas and products. 
This proliferation of research may have its benefits, 
but there are also limitations. School districts are 
barraged with products and services marketed as 
research-based, and many have limited time and 
capacity to ferret through those claims and judge 
their credibility. The problem is exacerbated in 
some places by diminished research departments 
and capacity during the recession. As IES strives to 
increase the relevance of research, it must continue 
to produce nonpartisan work and to communicate 
it as an honest broker. 

In the last decade, federal policy has put research 
front and center in education reform. ESRA set 
federally funded education research on a promising 
path, and reauthorization of it can take the field further 
down the path to relevance and impact. To do that, 
we will need to address a few issues: focus on state 
and local decision-makers as research users, address 
the questions relevant to them, and ensure rigor in 
studying those questions. It will also require a stronger 
infrastructure that connects research and practice, 
progresses from dissemination to engagement, and 
maintains IES’ role as an honest broker. 


' Coalition for Evidence Based Policy (2013). Randomized Controlled Trials Commissioned by the Institute of Education Sciences Since 2002: How Many Found 
Positive Versus Weak or No Effects. Retrieved from http://coalition4evidence.org/wp-content/uploads/2013/06/IES-Commissioned-RCTs-positive-vs-weak-or- 
null-findings-7-201 3.pdf. 

2 Coburn, C., Penuel, W. & Ceil, K. (2013). Research-Practice Partnerships: A Strategy for Leveraging Research for Educational Improvement in School Districts. 
New York, NY: William T. Grant Foundation. Retrieved from http://www.wtgrantfoundation.org/publications_and_reports/browse_reports/research-practice- 
partnerships-a-strategy-for-leveraging-research-for-educational-improvement-in-school-districts. 

3 Institute of Education Sciences (201 3, May). Program Announcement: Partnerships and Collaborations Focused on Problems of Practice or Policy CFDA 84.305H. 
Retrieved from http://ies.ed.gov/funding/ncer_rfas/partnerships_colab.asp. 
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New Directions in Education Research: 
The Federal Role 

Robert E. Slavin 
Johns Hopkins University 


The United States spends more on education than 
most developed nations, but levels of achievement of 
American children remain mediocre by international 
standards. On the National Assessment of Educational 
Progress, reading scores have hardly improved since 
1980, and math scores have made only modest gains. 
Achievement gaps by race and social class have hardly 
diminished. All of this is true despite constant federal 
and state tinkering with accountability schemes and 
other large-scale policy innovations. 

How can America’s schools become the best in the 
world, and solve longstanding problems ot equity and 
excellence? The answer lies in creating, rigorously 
evaluating, and then broadly implementing effective 
programs and policies. Research and development, 
followed by policies supporting the use of proven 
approaches, is essential to any strategy for impactful, 
sustainable gains in educational effectiveness on a 
national level. 

The Federal Role in Research 

The vast majority ot high-quality research in education 
is funded by the tederal government. Yet, compared 
to other fields such as medicine or agriculture, federal 
investment in educational R&D is miniscule. The U.S. 
Department ot Education’s Institute of Education 
Sciences (lES) spends about $239 million annually 
on R&D, compared to $31 billion at the National 
Institutes of Health alone (which is only one of many 
federal sources for medical research). This equates to 
roughly 2 days’ worth of NIH funding. Yet the federal 
investment is decisive, dominating all other federal and 
non-federal sources. Since lES was founded in 2002, 
it has moved education research rapidly toward a far 
greater use ot experimental evaluations of promising 
programs. Though lES also funds correlational studies, 
measure development, and other types of research, its 
flagship initiatives involve identifying potentially powerful 
programs and moving them from development to large, 
randomized evaluations. 


“The Insitute of 
Education Sciences' 
annual budget equates 
to roughly 2 days' 
worth of NIH funding." 

Another more recent source of education R&D within 
the Department of Education is the Investing in 
Innovation (i3) grant program, which is housed within 
the Department’s Office of Innovation and Improvement 
(Oil). The i3 program, which began as part of the 
Obama Administration’s stimulus efforts, focuses on the 
development and evaluation of promising programs, 
but adds an emphasis on dissemination of programs 
that already have strong evidence. 

The Department keeps track of programs through the 
What Works Clearinghouse (WWC), which summarizes 
evidence on programs that have been vetted through 
high-quality experimental evaluations. To add meaning 
to experimental evaluations, definitions of “strong,” 
“moderate,” and other levels of evidence were 
recently added to the Education Department General 
Administrative Regulations (EDGAR). Although many 
promising programs have not fared well in rigorous 
evaluations, an increasing numbers of programs with 
promising outcomes are being added to the WWG each 
year. 

The Education Sciences Reform Act 
(ESRA) 

A key underpinning for the new emphasis on evidence 
from rigorous evaluations is the Education Sciences 
Reform Act or ESRA, which is currently being considered 
for reauthorization. ESRA only applies to lES, not to i3 
or to other education R&D elsewhere in government, 
but it is essential because it sets a standard that others 
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increasingly follow. Some key suggestions for ESRA 

and for related policy improvements are as follows: 

1 Maintain the strong support in ESRA for randomized 
experiments, when possible, or at least high-quality 
quasi-experiments, when random assignment is 
not possible, in evaluating programs and policies. 
Current ESRA language to this effect should be 
maintained as it is. 

2 Designate lES as the lead office for evaluation of 
programs throughout the Department. Whenever 
outcome evaluations are carried out anywhere in 
the Department, lES should be involved to improve 
coordination among the various evaluations and to 
ensure that Department evaluations are performed 
with both scientific rigor and independence. 

3 Seek opportunities to do inexpensive, high-quality 
studies of innovative approaches using Department 
funding. Often, it is possible to do inexpensive 
studies using routinely collected data, such as 
state test scores or graduation rates, but such 
studies need to be designed and put into place in 
advance as part of the design of the federal grant. 
lES should be involved at an early stage in such 
impact evaluations to take advantage of these 
opportunities. Other offices within the Department 
of Education should be required to coordinate with 
lES to embed rigorous outcome evaluations in the 
design of Department grant competitions and other 
initiatives as a matter of course. 

4 Increase funding for all forms of education R&D. 
The large, randomized evaluations carried out 
under lES and i3 funding are expensive, and this 
greatly restricts the number that can be done. Also, 
correlational and observational research that leads 
up to development and rigorous evaluation is much 
needed. Funding levels for all of education R&D, 
especially in lES and i3, must be substantially larger 
to truly move the system toward the use of proven 
practices on a scale that matters. 

5 Encourage the use of proven programs in all 
Department grant funding. Whenever possible. 
Department programs should encourage use of 
programs that meet the "strong” or “moderate” 
levels of evidence in EDGAR. They may offer a 
few competitive preference points in competitive 
initiatives, or they may create funding streams or 
matching funds restricted to schools or districts 
willing to adopt proven programs. Also, clearing away 
hurdles to adopting proven programs is essential. 


For example. Title I schools should be assured that 
nothing in the “supplement, not supplant” rule should 
keep them from adopting proven programs. Another 
example would be to permit Title I funds to be used 
as matching funds for implementing i3 programs. 

Conclusion 

Research, evaluation, and dissemination of effective 
approaches should be the cornerstone of progress 
in America’s elementary and secondary schools. The 
current language in ESRA creates a good foundation 
for enabling lES to continue to play its leading role in 
this process. More funding, more integration with 
other Department of Education grant processes, and 
more emphasis on seeking cost-effective evaluations 
can increase the impact of research on educational 
practices and student outcomes. 
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The Evidence-based School District 

A Make-Believe District with a Real-World Mission 

MichaelJ. Petrilli 
Thomas B. Fordham Institute 


Why do many good ideas in education get overiooked? 
Why don’t research-based practices come to dominate 
the sector? Why don’t the very best curricular materials 
get adopted nationwide? 

If you asked me that question twenty years ago, I would 
have given a pat answer: incentives, or the lack thereof. 
In our bureaucratic education system, described 
most accurately as a public monopoly, nobody faced 
strong incentives to look for ways to build a better 
mousetrap. And if that mousetrap threatened anyone 
(as mousetraps tend to do), forget about it; the status 
quo ruled. 

Change the incentives and watch schools embrace 
change, I used to argue. Hold superintendents, 
principals, and teachers to account for raising tesf 
scores. Subject them to real competition. Then 
voila: They would spend night and day looking for 
promising innovations and proven practices to improve 
achievement and better serve families. 

Well, we know how that’s turned out. We’ve put a 
lot of those incentives in place, and made promising 
innovations easier to find (via the What Works 
Clearinghouse, the Regional Educational Labs, and 
other vehicles) and schools still don’t always embrace 
good ideas, even the non-controversial, inexpensive 
kind. Take, for instance, the following: 

• Bringing “departmentalization” to elementary 
schools by asking strong math teachers to teach 
math and strong reading teachers to teach reading, 
rather than expecting a single person to do both. 

• Maintaining a content-rich curriculum in elementary 
schools. E.D. Hirsch and others have demonstrated 
for decades that the best way to raise reading 
scores is to make sure students build a strong 
vocabulary and a strong knowledge base; elsewise, 
they won’t comprehend what they’re reading. Yet 
schools nationwide have pushed aside science 


and social studies, art and music to make room 
for mega-ELA blocks dominated by a focus on 
“reading skills.” 

• Extending the reach of excellent teachers via larger 
class sizes (with greater pay), new roles for master 
teachers, or technology. (Public Impact is chock-full 
of revenue-neutral ideas on this front.) 

• Delaying the start of the high school day to allow 
adolescents to get more sleep. 

“We’ve put a lot of 
incentives in place, 
and schools still don’t 
always embrace 
good ideas. What 
gives?” 

What gives? Some economists would argue that the 
incentives we’ve put in place to date aren’t strong 
enough. Even now, few educators lose their jobs if 
test scores don’t rise. Principals and teachers don’t 
generally stand to make much more money if they 
achieve breakthrough results (or attract gobs more 
customers). And competition, at least in most locales, 
is still quite limited. 

Or perhaps we’ve failed to get the details right. In 
most states, vague content standards, cheap, low- 
level tests, and incredibly low “proficiency cut scores” 
conspired to push schools toward basement-level 
instruction. Maybe now, with the Common Core and its 
next-generation assessments, we’ll finally send fhe right 
signals to practitioners, and they will respond by looking 
for concrete ways to improve instruction. 

All plausible. But there could be something simpler 
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at work: Perhaps many educators have never even 
encountered many of the ideas in question. Principals 
and teachers are so busy with the day-to-day struggle 
of their jobs — and now with new demands brought 
on by Common Core, new evaluation systems, and 
more — that they just keep their heads down and try to 
survive. They don’t have the time — or take the time — 
to read journals or blogs, to look for new innovations, 
to taik to colieagues, or to wonder about better ways 
of doing things. In this view, we have an “innovation- 
dissemination” (or “research-to-practice”) challenge. 

I’ll admit, that sounds like a bit of a cop-out, especially 
tor superintendents and principals. The leader of any 
organization knows that part of his or her job is to 
look for better ways to do things and to stay current 
on trends in the field. We should expect no less from 
our educational leaders, and those without an innate 
curiosity and drive for continuous improvement should 
be screened out of the profession. 

But these actors do face an avalanche of information 
and advocacy from the government, from think tanks, 
and especially from vendors. Sifting through it all 
and turning the best bits and pieces into a coherent 
approach is no easy task. (And this has been a problem 
forever.) 

Could we make that task more manageable? Could 
we help principals and superintendents to separate the 
wheat from the chaff in terms of the ideas that come 
across their desks on a given day? 

Introducing the Evidence-based Schooi 
District 

Imagine the creation of a make-believe school district. 
It wouldn’t have any actual students, teachers, 
buses, or facilities, but it would have a school board, 
a superintendent, and a central office staff. (The 
superinfendenf and staff would be paid real salaries and 
be housed in a real office; fhe school board would be 
made up of various “education experts” who, like real 
school board members, would volunteer their time.) It 
would be given a demographic profile — say, an inner- 
ring suburban district of 1 0,000 with a fair amount of 
racial and socioeconomic diversity. It would inherit the 
student achievement results, policies, and practices of 
a typical district. We’d situate it in an actual state, too. 


This simulated “school district” would be charged 
with developing and constantly updating a strategy 
for improving achievement and otherwise addressing 
the needs of its fictitious students. The board and 
superintendent might start by laying out a schedule for 
the year in which they would look at different key topics 
every month. Obvious contenders include: 

• An overall strategic plan for the district 

• An instructional plan focused on implementing the 
Common Core (including the adoption of aligned 
textbooks and other curricular materials) 

• A new educator evaluation system 

• A strategic approach to teacher recruitment, 
selection, and induction 

• A plan for addressing students’ social and 
emotional Learning 

• A strategy for integrating technology with fhe 
district’s instructional approach 

• A plan for improving the district’s special education 
services and outcomes 

• A set of policies on student discipline 

• An annual budget 

The “central office” would have staff working in roles 
similar to real districts: an assistant superintendent for 
curriculum and instruction, a chief human resources 
officer, a budget director, etc. These people might serve 
a faux district, but they would develop real plans to 
present to their superintendent and board. And — this 
is key — they would have tremendous resources to tap 
in helping them put these plans or policies or budgets 
together. Namely, they would have a big research budget 
and/or access to professionals at a think tank like the 
American Institutes for Research and/or the regional 
labs to help them sift through all of the relevant research, 
ideas, promising practices, and vendor pitches. 

The Potential of the EBSD 

Imagine what might happen if such a “school district” 
took off. First, it would develop policies, procedures, 
and plans that would be as robust as technocratically 
possible — aligned with the most sophisticated research 
and thinking available. These policies, procedures, 
and plans could then be swiped (and adapted) by real 
school districts for their own use. 
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“Imagine an ‘Evidence- 
based School DistricV 
that would develop 
policies, procedures, 
and plans that real 
districts could adapt 
for their own use. ” 

Second, it would provide small vendors of excellent 
products, think tankers with promising ideas, and 
advocacy groups brimming with sound suggestions 
with a national platform by which to spread the word. 
Everyone would know that if you wanted your policy or 
nostrum or solution to spread, you had to convince the 
EBSD that it was worth embracing. 

Of course, this would only work if actual school board 
members, superintendents, central office staff, and 
principals knew about it and found it helpful. It would be 
critical to get them involved — not just as recipients of 
the “content” produced by the EBSD but as producers 
themselves. (This would, in effect, make it “open 
source.”) They could join digital communities with others 
in their roles (such as HR managers) and interact with 
EBSD staffers as they develop their work products. If 
practicing educators had an idea or product or policy or 
practice that worked, they could share it with the virtual 
educator — and thus, the entire network. They could 
also watch the school board in action via a live stream 
or after the fact via video. 

At minimum, an EBSD would be a fascinating experiment 
and would probably produce some excellent materials. 
It wouldn’t be perfect — every state is different, for 
instance, so real-live district folks would have fo adapt 
materials and approaches for their own contexts. 
Furthermore, there would be no way to replicate the 
true push and pull of local politics with which real 
districts must contend. (How to come up with a model 
teacher union contract, for instance, in the absence of 
a teacher union?) One could also imagine all manner of 
lobbying, cajoling, and political pressure being placed 
on this fake district if its decisions affected the “real” 
marketplace. Board members and staff would need to 
be chosen carefully. Everything would need to be totally 
transparent. 


I suspect that those of us in the idea-generating 
business would be sobered by the experience. We 
would gain a better appreciation of the huge amount 
of conflicting advice and pressure that school districts 
and their leaders face. In fact, the most interesting part 
of the experiment would be seeing how the EBSD 
handles competing priorities and a policy environment 
that is anything but coherent. It might help us better 
understand which state and federal policies are helping 
districts improve and which are getting in the way. 

And yes, this assumes that school districts still have 
a major role to play for the foreseeable future. While 
that may not be the case in some of our big cities, 

I think the familiar structures will endure throughout 
most of the country and its suburbs and small towns. 
And it’s the small- to medium-sized districts — 
which serve nearly half of the nation’s public school 
students — that could benefit the most from this 
initiative, as they don’t have the scale to have much 
central-office capacity on their own. 

“IVs the small- to 
medium-sized districts 
that could benefit the 
most from this initiative, 
as they don’t have the 
scale to have much 
central-office capacity 
on their own.” 

First Steps 

How might we turn this nugget of an idea into reality? 

1 Build support for the EBSD among its potential 
users. There’s no point creating yet another 
organization that’s charged with synthesizing and 
disseminating information that the field ignores. 
Step one, then, is to determine whether there’s 
an appetite for this sort of a simulation — and its 
products — among superintendents, school board 
members, central office sfaff, and principals. The 
buy-in of the respective membership organizations 
for these officials will be essential. 
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2 Determine how to design inclusive processes 
from the start. If the ESBD’s “customers” 
demonstrate interest, the next step is to figure 
out how to make the simulation as accessible, 
transparent, and open-source as possible. The 
assumption is that educators will be more likely to 
use the materials produced by the ESBD if they 
have a role in creating them. Aggressive marketing, 
thoughtful use of technology, and well-designed 
processes will all be essential. 

3 Hire a top-notch staff and recruit a well- 
respected “school board.” The members of the 
staff should come from the field, will decades of real 
world experience, so as to make the simulation as 
realistic as possible. They also need to be well versed 
on the research base, committed to improving 
student achievement for all students, and open to 
working in a fishbowl. The school board members 
should represent the most thoughtful education 
experts available — with both an understanding of 
the evidence and a feel for real schools and systems. 

4 Develop a research budget. Determine how the 
staff and school board will be able to tap other 
experts for help as they do their work. Will a particular 
organization, like AIR, be put on retainer? Will each 
regional lab be expected to donate a certain amount 
of time? Or will the staff have access to financial 
resources which they can use as they see fit? 


5 Develop the EBSD’s core processes. These will 
include an annual process for selecting key work 
products to be developed; a hearing schedule to 
allow for public input; a vetting process to apply 
a rigorous approach to selecting products or 
approaches; and determination of how the “school 
board” will make its decisions. (By consensus? By 
majority vote?) 

America’s education system is famously fragmented, an 
“uncoupled train” that struggles to move in a straight 
line. That’s going to be the case as long as we have 
13,000-odd school districts — in other words, probably 
forever. But perhaps the creation of one more “school 
district” — the Evidence-based School District — would 
help bring a little more coherence, and a little greater 
quality, to our enterprise. 
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